Method and system for integration of portable devices with flight deck displays

ABSTRACT

A method for integration of a portable electronic device (PED) with a flight deck display in some embodiments includes providing an onboard network coupled to a flight-deck-installed, multi-function display and coupling the PED to the onboard network. The method may also include receiving information from a PED hosted application in the PED, routing information through a gateway and/or a plurality of firewalls, and displaying information on the flight-deck-installed, multi-function display. In some embodiments, a pilot control interface is coupled to the PED through the onboard network, and a plurality of functions on the PED is controlled via the pilot control interface. An airplane in some embodiments includes a fuselage portion, a wing portion, and a system that carries out the method, possibly through use of computer readable storage medium with stored instructions.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/054,788 filed Feb. 26, 2016, now U.S. Pat. No. 9,980,298, issued May22, 2018, the contents of which are incorporated herein by reference inits entirety.

FIELD

This invention related to airplanes and more particularly to the use offlight deck multifunction displays to distribute various kinds ofinformation to the flight crew of an airplane.

BACKGROUND

Modern commercial airplanes include numerous avionics display systemsand electronic control systems. The use of such systems is regulated andapproved by various governmental authorities around the world. Suchsystems are classified by these regulatory authorities according to thehazard level presented to an aircraft in flight if a system fails. As anexample, a system might be certified as a “Level A” certified system iffailure of the system would be very serious or catastrophic. A systemmight be classified as a “Level D” certified system if failure of thesystem would present a minor hazard that could be dealt with reasonablyby a properly trained and alert crew. Level B certified systems andLevel C certified systems fall between these two extremes.

Some systems on an aircraft, such as an electronic flight bag (EFB) maynot be certified. These systems are typically those added by or for aspecific airline and are not part of the basic, factory-provided flightsystems. A governmental authority also gives approval for these systemsto be used operationally during flight but there are less stringentdesign requirements because a failure of one of these systems has noimpact on the safety of flight. The result is an easier path to iteratethe design and evolve the functionality over time without therequirement of an expensive re-certification effort. Many of theseuncertified systems provide supplemental information to the crew, suchas dispatch routing, planning and payload information.

Over the past decade, there has been a significant shift in how suchinformation is consumed on the flight deck. Manuals, flight plans,charting information, performance tables, etc. that assist the flightcrews in performing their overall mission but are not critical to theoperation of the airplane were provided in paper form for many years.However, in the current technological and economic environment,flexible, uncertified, tablet-based computing systems are being used toprovide this information to pilots. Such a system is referred to as aportable electronic device (PED) based EFB. This format allows airlinesto more easily realize the weight and handling cost savings associatedwith eliminating paper-based information. In order to be used during allphases of flight, however, these PED-based EFB systems must be securelymounted in the flight deck. Unfortunately, in some flight decks, thereis not sufficient space or a suitable location to mount a PED for easyviewing and interaction. In addition, it is sometimes difficult toposition a tablet computer for shared viewing or convenient viewingalongside flight deck multifunction displays resulting in more difficultcrew coordination reducing the overall usability of the PED-based EFBinformation.

SUMMARY

In at least some embodiments of the invention, a method of accessing aportable electronic device (PED) includes providing an onboard networkcoupled to a flight-deck-installed, multi-function display and couplingthe PED to the onboard network. In such an embodiment, the method alsoincludes receiving information from a PED hosted application in the PED,routing the information through a gateway, and displaying theinformation on the flight-deck-installed, multi-function display. Insome embodiments, a pilot control interface is coupled to the PEDthrough the onboard network, and a plurality of functions on the PED iscontrolled via the pilot control interface.

In at least some embodiments of the invention, an airplane includes afuselage portion and a wing portion. The airplane also includes aflight-deck-installed, multi-function display, an onboard network, asecurity gateway connected to the onboard network, and an aircraftinterface device (AID) connected to the onboard network to couple aportable electronic device (PED) to the onboard network. The aircraftinterface device is operable to receive information from a PED hostedapplication through a gateway and route the information through agateway or gateways to the airplane's avionics. Avionics then displaysthe information on the flight-deck-installed, multi-function display.The airplane in some embodiments can also include an aircraft datanetwork used by certified avionics for communication between certifiedcomponents, a flight deck crew network accessible by operationallyapproved, un-certified applications hosted on the PED wherein anapplication gateway on the AID routes information between the aircraftdata network and the flight deck crew network through a securitygateway. In some embodiments, a plurality of firewalls isolates theaircraft data network and the flight deck crew network. The aircraftdata network and flight deck crew network can alternatively be describedas an aircraft control domain and aircraft information domain,respectively, where the aircraft information domain has a lower securityand certification level than the aircraft control domain.

In at least some embodiments of the invention, a system includes aflight-deck-installed, multi-function display, and an onboard network,which may further include a flight deck crew network and an aircraftdata network. The system can also include the security gateway, theaircraft interface device, and avionics systems connected to the onboardnetwork and the flight-deck-installed, multi-function display to displaythe information on the flight-deck-installed, multi-function display.Instructions stored on a computer readable storage medium can beprovided to, or stored within the system, wherein these instructionsestablish the onboard network coupled to a flight-deck-installed,multi-function display, for example, by providing the appropriateprotocol stacks. The instructions can also couple the PED to the onboardnetwork, receive information from a PED hosted application in the PED,route the information through a gateway or gateways, and display theinformation on the flight-deck-installed, multi-function display, aswell as provide for other functions and/or features described herein.

The features, functions, and advantages that have been discussed can beachieved independently in various embodiments or may be combined in yetother embodiments further details of which can be seen with reference tothe following description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of a portion of the flight deck of an airplanein an exemplary embodiment of the invention.

FIG. 2 is an illustration of an airplane in an exemplary embodiment ofthe invention.

FIG. 3 is an illustration of hardware and/or software entities in anexemplary embodiment of the invention.

FIG. 4 is a schematic illustration of flight deck display screens in asystem in which an embodiment of the invention is in use.

FIG. 5 is an illustration of operations performed by one embodiment.

FIG. 6 is an illustration of operations performed by another embodiment.

Each figure shown in this disclosure shows a variation of an aspect ofthe embodiments presented, and only differences will be discussed indetail.

DETAILED DESCRIPTION

Embodiments of the invention provide for the use of the main flight deckmulti-function displays to distribute portable information broughtonboard an airplane across multiple flight deck display surfaces. Thiscapability allows the crew of an airplane to take advantage of powerfuland flexible tablet devices to host uncertified information that can beintegrated with other information typically shown on the main flightdeck displays. This capability can improve the ergonomics of usingelectronic flight bag (EFB) or other applications hosted on a portableelectronic device (PED) by allowing utilization of larger displaysurfaces and by allowing crewmembers to more easily share information ina viewable location. Such capability may allow airlines to remove backupPEDs from the flight deck that were a contingency in case of devicefailure, resulting in cost savings. Crewmembers can also share jointattention to a common viewable display in order to promote good crewmanagement practices. Information can be distributed across multipledisplays in the flight deck to promote better situational awareness, andreduced workload associated with information search. Since largerdisplays are available for tablet-based information, smaller, lessexpensive devices such as smart phones can be used to displayinformation, again resulting in cost savings.

With at least some embodiments of the invention, the aircraft's networkis used to securely integrate graphical information directly from atablet application with airplane-hosted information. Images and/or videois/are output to the display avionics for presentation on the mainmulti-function flight deck displays. In some embodiments, the systemmakes crew inputs available to the tablet-hosted application to make thetablet-hosted application fully interactive on the main multi-functiondisplays. All interactions are mediated directly between the PED-hostedapplication and airplane-hosted systems eliminating any dependencies onspecific tablet hardware or operating systems. This arrangement makesthe system flexible and enables new features to be added easily.

This detailed description of embodiments refers to the accompanyingdrawings, which illustrate specific embodiments of the disclosure. Otherembodiments having different structures and operations do not departfrom the scope of the present disclosure. Like reference numerals mayrefer to the same element or component in the different drawings.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

FIG. 1 is a forward perspective view of an exemplary portion of anairplane flight deck panel 100 that includes at least one multi-functiondisplay head with screen 102 in accordance with an embodiment of thepresent invention. In the exemplary embodiment, the display screen ispositioned on airplane flight deck panel 100. In this view, a portableelectronic device (PED) 104 is also shown. A connection 106 has beenestablished between PED 104 and certified systems aboard the airplane,portions of which lie out of view of panel 100, for example, in anelectronics bay. During aircraft operation, screen 102 is available forviewing by a pilot and/or co-pilot of the airplane and can be made todisplay information hosted on PED 104. It is also possible for any orall of various pilot controls in pilot control interface 108 to becoupled to PED 104 to provide input to PED 104. Detailed examples ofspecific input and control devices that may be included in the pilotcontrol interface include a touchpad, a trackball, a rotary selectioncontrol (“tabber,”) a single-touch touchscreen, a multi-touchtouchscreen, a pressure sensitive touchscreen, a multi-function keypad,an MCDU scratchpad, a user eye tracking device, a voice activation/voicecontrol device, and a system automation input device. It is alsopossible for a remote control device or a control from a separate PED tobe coupled through the system as described herein.

FIG. 2 is a schematic illustration of an airplane 201 having a system200 configured to display information from a PED in accordance with anembodiment of the invention. Portions of the system 200 can be housed inor near a flight deck 240 of the aircraft 201 for access by acrewmember. Portions may also be located in service bays, racks, and thelike in other parts of the airplane, such as fuselage 205, which isconnected to wings 207. In one aspect of this embodiment, the system 200can include a flight-deck-mounted, multi-function display like thatshown in FIG. 1, a PED 104, and an aircraft interface device (AID) 210connected to a flight deck crew network (not shown) to establish aconnection with a portable electronic device (PED), wherein the AIDestablishes the connections to PEDs. The AID is operable to access acomputer readable storage medium 212 that provides executable computerprogram instructions 214 to implement an application gateway to receiveimage data from the PED over the flight deck crew network. The system insuch an embodiment also includes avionics 220 connected to the AID andincluding the flight-deck-mounted, multi-function display to receive theimage data from the AID and display an image or images corresponding tothe image data on the flight-deck-mounted, multi-function display. Theavionics can, in example embodiments, include a security gateway 230,wherein at least some pilot interaction data and/or image data and/orother information passes through the security gateway on its way toand/or from the PED. Computer readable storage medium 232 includesexecutable computer program instructions to create the security gateway.

All of the system components that are included within the AID andavionics illustrated in FIG. 2 are certified computing systems thatinclude a processor and other components typical of such systems andthus the system 200 as well as any of the component computing platformscan be referred to herein as a certified system. In the exampleembodiment of FIG. 2, image frames and/or the pilot interaction datapass from/to PED 104 and this communication to and from the PED ismanaged by an application gateway that instantiates from computerprogram instructions 214 on non-transitory storage medium 212 within theAID and from computer program instructions 216 on non-transitory storagemedium 219 within the PED. The functions provided by the applicationgateway can include, as examples, at least some of deviceauthentication, bandwidth monitoring, content analysis, enforcement ofregulatory policies, enforcement of and configurability to airlinepolicies, warning/alerting, operational status monitoring, failurehandling, intermediate formatting, and compositing of applicationinformation. Some of these functions can also or alternatively beprovided by the security gateway. The security gateway and the AID worktogether to provide multiple layers of network security. The term“gateway” as used herein can refer to either or both of the applicationgateway and the security gateway when describing different aspects ofthe disclosure.

FIG. 3 is a more detailed schematic system block diagram of system 200from FIG. 2, but with two PEDs 104 connected. In this example, each PED104 includes an application 216. As previously mentioned, the on-boardcomputing systems and applications are all certified. The PED and itsresident applications are uncertified but may be operationally approved.A PED application includes application logic and an off-screen framebuffer. The application also includes interfacing logic and anaccompanying command library for interfacing with the applicationgateway 304 in AID 210. The application gateway interface 350 includescontrol logic, a display selection function, an application selectionfunction, and management functions such as device and applicationauthentication, bandwidth monitoring, handshaking, a watchdog timer,failure handling and the like. The application gateway also includesintermediate image formatting and image conversion for standard digitalvideo interface (DVI) outputs. The application gateway in additionincludes one of many possible firewalls that can be included in thesystem to isolate the aircraft data network 211 and the flight deck crewnetwork 213.

PEDs 104 of FIG. 3 are connected to the AID through a flight deck crewnetwork, which provides for the transfer of information including imagedata, filtered pilot control or interaction data, and applicationhandshakes, which in part are used to establish a secure connectionbetween the PED and on-board systems. The interface in this example maybe either a wireless interface such as Wi-Fi or a standard wiredEthernet interface or connection. The interface can be implemented usingany secure wired or wireless connection, including a wireless interfacesuch as Bluetooth or NFC, or an avionics full duplex switched Ethernet(AFDX) connection.

Still referring to FIG. 3, AID 210 includes image formatting function309 within the application gateway 304, and image conversion 310, whichreceives image data from the formatting function. The purpose of the AIDis to protect certified systems from uncertified, typicallyoperationally approved systems. Image data can include data to providevideo or still images, or a combination of the two, including compressedvideo information. In the example embodiment shown, the conversion ishandled by converters implemented by field programmable gate array(FPGA) chips within the AID hardware. These converters take standardimage frame data (DVI format as an example) as input and converts theimage frames to image frames that can be passed over another standardvideo interface, as an example, an A818 avionics digital video bus(ADVB), which is part of the aircraft data network.

Two sets of image frame data are supplied to avionics 220 as shown inFIG. 3. One set of image frame data is supplied to graphics generatormodule 316 and another set of image frame data is supplied to graphicsgenerator module 318, either of which is capable of merging imagesand/or information from the PED with certified information from theon-board systems for unified presentation on the flight deck,multi-function display heads 320 and 322. Additional firewalls protectthe graphics generators. These firewalls are schematically indicated bydotted lines 328 and 329. These displays are pictured in FIG. 3 as partof the avionics 220 since they are part of the certified equipment onthe aircraft, however they are often connected via cables to the otheravionics hardware, which may reside in an equipment bay. Any of thesearrangements can be invoked by reference to the displays or displayheads being connected to or part of the avionics.

In addition to illustrating the flow of image frame data, FIG. 3schematically illustrates the sending of pilot interaction data fromflight deck systems to PEDs 104 over the established flight deck crewnetwork. In the example embodiment of FIG. 3, this control data passesfrom pilot controls 330 to displays logic 332, which manages both thefirewalled graphics generators in accordance with display configurationsettings received through the aircraft data network. In exampleembodiments, image data and pilot interaction data pass through thesecurity gateway 230. Displays configuration data can be passed from theapplication gateway to the displays logic and may include the status ofcertified display widgets, window management logic, and data layeringlogic. In some example embodiments, the graphical user interface can bedefined with stored binary files according to the ARINC 661 (A661)standard. Pilot interaction data that is ultimately sent to PEDs 104 caninclude cursor control data. Pilot interaction data can also includetouch interaction data, data on the position of rotary tabberinterfaces, and data on any other controls that the pilots canmanipulate. The AID is operable to send the pilot interaction data fromthe flight deck to the PED. Pilot interaction data in some embodimentsis filtered by the AID application gateway and/or the security gatewaybefore being sent to the PEDs.

As noted above, the system illustrated in FIG. 3 includes a plurality offirewalls to isolate the aircraft data network and the flight deck crewnetwork. The aircraft data network and flight deck crew network canalternatively be described as an aircraft control domain and aircraftinformation domain, respectively, where the aircraft information domainhas a lower security and certification level than the aircraft controldomain. A firewall monitors and controls a resource's incoming andoutgoing traffic based on predetermined security rules, and thus forms abarrier between a protected network element and other parts of thesystem. In a system like that shown in FIG. 3, firewalls or subsets offirewall functionality would be present on all connections betweencomponents of differing certification levels, protecting the device ofhigher certification. Firewalls or subsets of firewall functionalitywould also be present on all connections, which are “open” to orreachable from outside devices or systems that have outside deviceconnectivity (including off-board data links).

Firewalls or subsets of firewall functionality would be present on allcomponents, which process data originating from an open connection orlower-certification component, which has reached the current system bymeans of a non-inspecting pass-through by a system, which wouldotherwise not be firewalled. For example, PED applications may beoperationally approved and generate data which is received by certifiedsystems on the airplane. The certified systems that initially receivethe data do not use the data payload, but instead forwards data toanother certified system. Knowing that the first certified system didnot inspect the data payload, the second certified system applies asubset of firewall filtering to the data as it is unpacked but before itis used, even though the data was provided by another certified system.

FIG. 4 shows close-up, schematic views of screens 102 from FIG. 1, eachshown in detail as screen 102 a, screen 102 b, screen 102 c, screen 102d, and screen 102 e. The arrangement of screens deviates from that shownin FIG. 1 in order to show all screens on a single drawing sheet. Eachscreen is a visible portion of a flight-deck installed, multifunctiondisplay. For purposes of the discussion herein, the term “screen” andthe term “display” can be used interchangeably. With example embodimentsof the invention, a plurality of input and control devices and theflight-deck-installed, multi-function display screens are usable by aplurality of users. Each user can provide input and control his or herown PED and flight-deck-installed, multi-function display. Each user canprovide input and exclusively control an application on another user'sPED from their own input and control device and flight-deck-installed,multi-function display. Alternatively, each user can control separatePEDs independently or users can cooperatively control a single PED.

In FIG. 4, applications hosted with the displays can include a window onthe flight-deck-installed, multi-function display, such as window 402 ondisplay 102 b, indicated by cross-hatched lines. Applications can alsoinclude a window managed across multiple flight-deck-installed,multi-function displays such as application 404, indicated by diagonallines, managed across displays 102 c and 102 e. Multiple applications onmultiple devices can also be hosted with a diversity of pilot controlinterfaces, including an ability to operate without the pilot controlinterfaces. Compositing of certified information 406 and operationallyapproved information 408 (shown with stippling) is also shown on display102 a. Multiple user and shared/exclusive application environmentapplications are also supported. Device-based status informationincluding additional application categories can be sent to a gateway.Operationally approved information 410 on display 102 d shares some ofthe same information as shown in operationally approved information 408.

Still referring to FIG. 4, information sent to the avionics can be fromapplications with additional application categories. These applicationscan include a hosted application with an abstract nature, a non-videoapplication, a function one or more of video displays 102 a, 102 b, 102c, 102 d, and/or 102 e. The PED can utilize the flight-deck-installed,multi-function display as a passive display terminal without anyrequirement for display or graphics generation/output hardware, or anynative forms of input or interactivity within the PED. The function ofthe video display can provide a service to directly connect an abstractapplication for display on and interaction with a flight-deck-installed,multi-function display, regardless of specifics, services, capabilitiesor operating system of the PED.

An “abstract application” is an application that does not fit thetraditional concept of what a user expects an “application” to be.Examples of a traditional application include one that is launched ondemand, generates graphics or present imagery in a rectangular view on adisplay surface, and is controllable or interactive in some way such asby direct touch interaction, manipulation by a mouse/touchpad drivencursor, etc. Abstract applications are often not called up on-demand,serving as more of a “function” than an “application.” As examples, anapplication which runs in the background, waits for data from anexternal input, and sends that data to a display as a notification couldbe considered an abstract application. Applications that are notinteractive can be considered abstract applications. Applications thatrun without displaying content, whether or not they are interactive, maybe considered abstract applications. Applications, which create displaysof irregular shapes or sizes, such as one that creates arc-shapedweather radar imagery may be abstract applications. Applications whichare launched on demand without the end user being aware of theirexistence, serving as more of a “service” than an “application” can beconsidered abstract applications.

As further examples of abstract applications, an application, whichrenders a layer or contributing piece of imagery, which is ingested,processed, and presented to the user as part of a different applicationor function would be an abstract application. As a more specificexample, an application which retrieves and renders weather imagery,which is handed off and then used by a pilot-facing navigation displaythat doesn't have native weather retrieval capabilities is an abstractapplication. Applications that present a display and are interactive,but whose sole purpose is to influence a different system or applicationare abstract applications. As an example, a “remote control” type ofapplication, such as an emulated version of a flight deck physicalbutton control panel can be considered an abstract application.Applications, which run in the background to facilitate connectivitywith the aircraft or the forward display system are abstractapplications.

FIG. 5 is an illustration of operations performed by one embodiment. Thesequence of operations for the process is illustrated in flowchart form.Process 500 begins at start block 502. At block 504, a handshake takesplace between the PED(s) and the AID. At block 506, a PED hostedapplication is coupled to the on-board network, more specifically inthis example to the application gateway in the AID. A connection is nowestablished between the PED(s) and the certified systems on theairplane. Information from the PED(s) and pilot interaction data aresent at the same time. At block 508, pilot interaction data isselectively sent to the PED(s) from the pilot control interface. Sendingpilot interaction data is further described below with respect to FIG.6. By “selectively sent,” what is meant is that the crewmembers and/orthe avionics and/or the application gateway can control which data issent—not every pilot interaction is necessarily sent to a given PED. Forexample, there may be multiple PEDs connected and any PED may havemultiple applications running. Selectivity can include sendinginteraction data only to a specific PED or PEDs or only to a specificapplication in a specific PED or PEDs rather than to all applicationsrunning in a PED or all PEDs.

At block 509 of FIG. 5, the application gateway provides any or all ofmany services. These services can also be provided by the securitygateway, or the services can be distributed across both gateways. Atblock 510, the certified AID receives information from the PED(s) underthe control of the AID application gateway. This information may includeimage frames for images to be displayed on the flight deck. The servicesprovided by the gateway can include device authentication, bandwidthmonitoring, device and network control, content analysis, enforcement ofregulatory policies, enforcement of and configurability to airlinepolicies, warning/alerting, operational status monitoring, and/orfailure handling. Additional services provided by the applicationgateway can include management of connected devices, applicationselection, intermediate formatting, compositing of applicationinformation, application authentication, filtering of devices and/orapplications, which can or cannot communicate with the system, and/orinterfacing and data translation between disparate system components.The application gateway can also relay aircraft information data andpilot interaction control data, enforce compliance with flight deckhuman interface standards and guidelines, enforce airplane manufacturerpolicies and acceptability as per an established application governanceprocess and/or integrate applications hosted on operationally approvedPEDs with certified or operationally approved applications hosted withinthe onboard network. The application gateway can also providepreparation of information from the PED for integration and/or layeringwithin functional layered applications on a certified displayapplication.

Still referring to FIG. 5, at block 512, image data or other informationis converted as necessary. At block 514, the information is forwardedfrom the AID to the avionics. This information traverses the securitygateway which may be capable of performing all or some of the functionsattributed to the application gateway.

Staying with FIG. 5, at block 516, as already described, firewalledgraphics modules in the avionics can selectively display an image orimages corresponding to the information received from the PED(s) on amulti-function display installed in/on the flight deck of the airplane,through the coupling between the PED and the avionics. The phrase“selectively display” means pilots and avionics functions can controlwhat information is displayed and how the information is displayed. Forinstance, pilots may choose to display image data from an application infull screen or half screen format, or pilots can choose to replace thePED image with an image in another certified display format. In afailure condition or emergency, the displays logic may exclude thedisplay of operationally approved information on the multi-functiondisplays reserving available space for the more important certifieddisplay formats.

Optionally, information from certified and uncertified systems isselectively merged together as needed for display. By “selectivelymerged” what is meant is that the crewmembers or the avionics candetermine how or if the different types of information are mergedtogether. A display can contain information from both certified andnon-certified sources. How this information is merged and/or how each isindicated can be pre-programmed within the certified avionics or managedby the application gateway through the forwarding of displaysconfiguration settings to the displays logic. Control over whatinformation to display could be automated or manually selectedindependently be each pilot. Information can be overlaid, or layered tokeep certain information visible or give priority to certaininformation. If at some point during the process of FIG. 5 a PED isdisconnected at block 518 of FIG. 5, the process ends at block 520, atleast relative to that PED. Otherwise, process 500 repeats as long asthe PED is connected to the flight deck systems.

To elaborate on some of the services that can be provided by theapplication gateway, device authentication in at least some embodimentsincludes validating that a device is authorized to communicate with thesystem. Application authentication, includes validating that anapplication is authorized to communicate with the system. Bandwidthmonitoring includes monitoring communications to ensure that devices areusing the available bandwidth to the fullest benefit, but not to theextent of crowding out other network services. Device and networkcontrol provides the ability to manage a PED's interactivity with thenetwork, including removing the device from the network. Contentanalysis can include rule-based inspection of imagery and data beingpassed through the application gateway. As an example, the enforcementof regulatory policies might include verifying that applications havebeen operationally approved and accepted for inclusion as part of agovernance process by the airplane manufacturer or operating entity, andthat applications that have not been approved are not allowed access. Anexample of enforcement of and configurability to airline policies, mightinclude preventing personally owned PEDs from being connected to thesystem.

For warning/alerting, the application gateway can provide a mechanismfor applications to draw attention to themselves in situations requiringpilot action. For operational status monitoring, the “health” of anapplication can be monitored to determine its readiness state andperformance. The application gateway can provide failure handling bytaking action appropriately in the event that one or more systemcomponents malfunctions or does not perform as expected. The applicationgateway can manage connected devices by correlating, listing,displaying, filtering, and assigning of the connected devices. Portionsof these services are system tasks, and portions of these services arebased on pilot actions. Application selection can be provided bypresenting applications in a user interface for selection and activationby the pilot or other user of the system.

The application gateway can provide intermediate formatting byappropriately managing application imagery, which is not an exact matchfor the display on which it is presented. This intermediate formattingcan include “physical” transformations such as scaling, cropping,rotation, positioning, etc., as well as image data manipulations such asthose related to color saturation, contrast, sharpening, etc. Thecompositing of application information includes combining information(imagery and/or data) from one or more PEDs with information orgraphical display elements from one or more other sources. To filterdevices and/or applications, which can or cannot communicate with thesystem, applications must know the appropriate communication protocolsto interface with the system. Those that do not “speak the rightlanguage,” or which have incompatible or discontinued/outdated versionsof the protocols will be refused the connection or have theirconnections terminated. The application gateway can provide interfacingand data translation between disparate system components by serving asthe central hub of data management for a multitude of separate systems.This service can include retrieving data, processing data, and/orforwarding data in formats, which are appropriate for the device,system, or application that is consuming the data. The applicationgateway can relay aircraft information data and pilot controlinteraction data by acting as a forwarding bridge to send aircraft dataparameters (such as flight management information) and pilot interactioncontrol data (such as touchscreen touch events) from the aircraftsystems to a PED through the AID.

The application gateway can also provide for enforcement of andcompliance with flight deck human interface standards and guidelines. Asan example, these policies may specify the approved color palette foruse in flight deck displays. The enforcement of airplane manufacturer orregulatory policies and acceptability as per an established applicationgovernance process might include, as an example, prohibitingentertainment applications on the forward displays. The integration ofapplications hosted on operationally approved PEDs with certified oroperationally approved applications hosted by the AID may includecompositing and screen sharing of parallel functions from differentsources and certification levels. The preparation of information from aPED for integration and/or layering within functional layeredapplications on a certified display application may include processing,formatting, geo-referencing, compositing, and/or selective display ofinformation layers (data or imagery), which will contribute to acomposite representation of multi-sourced data/imagery on a forwarddisplay.

FIG. 6 is an illustration of operations performed by one embodiment. Thesequence of operations for the process corresponds to block 508 of FIG.5 and it is again illustrated in flowchart form. It should be noted thatthe process of allowing crew control of a PED includes informationmoving two ways. Display configuration settings and similar informationrelated to specifying the display interface is sent to from theapplication manager to the avionics in order to define theconfigurations of flight deck controls, while the pilot interaction datarelated to accessing the PED is sent from the pilot control interface tothe PED over the established connection through the relevant networks,the aircraft data network and the flight deck crew network. Displayconfiguration settings information are formatted as requests. Theserequests can be acted on by the avionics or not in accordance withpolicies that are programmed into the avionics, for example, as part ofthe security gateway or displays logic functions as described above.Process 508 starts at block 602. At block 606, graphical displayconfiguration data is received by the avionics through the securitygateway from the AID application gateway. At block 608, the avionicsforwards pilot interaction data to the application gateway in the AID.At block 610, pilot interaction data are filtered prior to being sent tothe PED. Filtering can include, as examples, device filtering andapplication filtering, and policing relative to various regulatory orairline policies as described above. Device and application filteringensure that the correct pilot input data get to the right device andapplication. At block 614, pilot interaction data is sent from the AIDto the PED through the onboard network to control PED functions from thepilot control interface in the flight deck of the airplane. Process 508ends at block 616, although in the context of the larger process shownin FIG. 5, process 508 repeats just as portions of the process of FIG. 5repeat.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It is also noted that each block of the blockdiagrams and/or flowchart illustration, and combinations of blocks inthe block diagrams and/or flowchart illustration, can be implemented byspecial purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The terminology used herein is for the purposes of describing particularembodiments only and it is not intended to be limiting of embodiments ofthe invention. As used herein, the singular forms “a”, “an” and “the”are intended to include the plural forms as well, unless the contextclearly indicates otherwise. It will be further understood that theterms “comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to embodiments of the invention in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of embodiments ofthe invention. The embodiment was chosen and described in order to bestexplain the principles of embodiments of the invention and the practicalapplication, and to enable others of ordinary skill in the art tounderstand embodiments of the invention for various embodiments withvarious modifications as are suited to the particular use contemplated.

Although specific embodiments have been illustrated and describedherein, those of ordinary skill in the art appreciate that anyarrangement which is calculated to achieve the same purpose may besubstituted for the specific embodiments shown and that embodiments ofthe invention have other applications in other environments. Thisapplication is intended to cover any adaptations or variations of thepresent invention. The following claims are in no way intended to limitthe scope of embodiments of the invention to the specific embodimentsdescribed herein.

The invention claimed is:
 1. A method of accessing a portable electronicdevice (PED) on board an aircraft, the method comprising: providing anonboard network coupled to a flight-deck-installed, multi-functiondisplay, wherein the onboard network comprises a flight deck crewnetwork and an aircraft data network; coupling the PED to the onboardnetwork; receiving information from a PED hosted application in the PEDby an aircraft interface device; routing the information through theaircraft interface device to the aircraft data network; receiving theinformation from the aircraft data network by a security gateway,wherein the aircraft interface device and the security gateway providemultiple layers of network security; displaying the information on theflight-deck-installed, multi-function display; and coupling a pilotcontrol interface to the PED through the onboard network, wherein thecoupling of the pilot control interface to the PED comprises coupling aplurality of input and control devices to the PED, the plurality ofinput and control devices and the flight-deck-installed, multi-functiondisplay are usable by a plurality of users where each user can provideinput and control their own PED and flight-deck-installed,multi-function display.
 2. The method of claim 1 further comprisingcontrolling a plurality of functions on the PED via the pilot controlinterface.
 3. The method of claim 2 wherein the plurality of input andcontrol devices includes: a touchpad, a trackball, a rotary selectioncontrol or tabber, (“tabber,”) a single-touch touchscreen, a multi-touchtouchscreen, a pressure sensitive touchscreen, a multi-function keypad,an MCDU scratchpad, a user eye tracking device, a voice activation/voicecontrol device, a system automation input device, a remote controldevice, a control from a separate PED, or a combination thereof.
 4. Themethod of claim 3 where each user can provide input and exclusivelycontrol an application on another user's PED on theirflight-deck-installed, multi-function display, where each user cancontrol separate PEDs independently, and where users can cooperativelycontrol a single PED.
 5. The method of claim 1 wherein the aircraftinterface device comprises an application gateway wherein theapplication gateway or the security gateway is operable to provide afunction comprising: device authentication, bandwidth monitoring, deviceand network control, content analysis, enforcement of regulatorypolicies, enforcement of and configurability to airline policies,warning/alerting, operational status monitoring, failure handling,management of connected devices, application selection, intermediateformatting, compositing of application information, applicationauthentication, filtering of devices and applications which can orcannot communicate with the onboard network, interfacing and datatranslation between disparate system components, relaying of aircraftinformation data and pilot interaction control data, enforcement of andcompliance with flight deck human interface standards and guidelines,enforcement of airplane manufacturer policies and acceptability as peran established application governance process, integration ofapplications hosted on operationally approved PEDs with certified oroperationally approved applications hosted within the onboard network,preparation of information from the PED for integration or layeringwithin functional layered applications on a certified displayapplication, or a combination thereof.
 6. The method of claim 1, whereinfor a function of video display, the PED utilizes theflight-deck-installed, multi-function display as a passive displayterminal without any requirement for display or graphicsgeneration/output hardware on the PED, or any native forms of input orinteractivity on the PED and wherein the function of video displayprovides a service to directly connect an abstract application to themulti-function displays via the gateway for display and interaction onthe flight-deck regardless of specifics, services, capabilities oroperating system of the PED.
 7. The method of claim 1, whereinapplications hosted with flight-deck-installed, multi-function displaycomprise: a window on the flight-deck-installed, multi-function display,a window managed across multiple flight-deck-installed, multi-functiondisplays, multiple application on multiple devices, a diversity of pilotcontrol interfaces including an ability to operate without the pilotcontrol interfaces, an abstract nature of hosted applications includinga non-video application, a compositing of certified and operationallyapproved information, a multiple user and shared/exclusive applicationenvironment application, device-based status information sent to agateway including additional application categories, or a combinationthereof.
 8. An airplane comprising: a fuselage portion; a wing portion;and a flight-deck-installed, multi-function display; an onboard networkcomprising an aircraft data network and a flight deck crew network,wherein the onboard network comprises a plurality of firewalls toisolate the aircraft data network and the flight deck crew network; agateway connected to the onboard network, wherein the gateway comprisesa security gateway; an aircraft interface device connected to theonboard network to couple a portable electronic device (PED) to theflight-deck-installed, multi-function display, the aircraft interfacedevice operable to receive information from a PED hosted application inthe PED and route the information through the gateway, wherein theaircraft interface device and the security gateway provide multiplelayers of network security and wherein the aircraft data network is usedby certified avionics for communication between certified components andthe flight deck crew network is accessible by operationally approved,un-certified applications hosted on the PED wherein the gateway routesinformation between the aircraft data network and the flight deck crewnetwork; and avionics to display the information on theflight-deck-installed, multi-function display.
 9. The airplane of claim8 wherein the flight deck crew network comprises a wired Ethernetinterface.
 10. The airplane of claim 8 wherein the flight deck crewnetwork comprises a wireless interface.
 11. The airplane of claim 8further comprising a pilot control interface wherein the aircraftinterface device is further operable to couple the pilot controlinterface to the PED through the onboard network to enable controlling aplurality of functions on the PED via the pilot control interface. 12.The airplane of claim 11 wherein the pilot control interface comprises aplurality of input and control devices including: a touchpad, atrackball, a rotary selection control or tabber, a single-touchtouchscreen, a multi-touch touchscreen, a pressure sensitivetouchscreen, a multi-function keypad, an MCDU scratchpad, a user eyetracking device, a voice activation/voice control device, a systemautomation input device, a remote control device, a control from aseparate PED, or a combination thereof.
 13. The airplane of claim 12wherein the plurality of input and control devices and theflight-deck-installed, multi-function display are usable by a pluralityof users where each user can provide input and control their own PED andflight-deck-installed, multi-function display, where each user canprovide input and exclusively control an application on another user'sPED on their flight-deck-installed, multi-function display, where eachuser can control separate PEDs independently, and where users cancooperatively control a single PED.
 14. The airplane of claim 8 whereinthe gateway is operable to provide a function comprising: deviceauthentication, bandwidth monitoring, device and network control,content analysis, enforcement of regulatory policies, enforcement of andconfigurability to airline policies, warning/alerting, operationalstatus monitoring, failure handling, management of connected devices,application selection, intermediate formatting, compositing ofapplication information, application authentication, filtering ofdevices and applications which can or cannot communicate with theonboard network, interfacing and data translation between disparatesystem components, relaying of aircraft information data and pilotinteraction control data, enforcement of and compliance with flight deckhuman interface standards and guidelines, enforcement of airplanemanufacturer policies and acceptability as per an establishedapplication governance process, integration of applications hosted onoperationally approved PEDs with certified or operationally approvedapplications hosted within the onboard network, preparation ofinformation from the PED for integration or layering within functionallayered applications on a certified display application, or acombination thereof.
 15. The airplane of claim 8, wherein for a functionof video display, the PED utilizes the flight-deck-installed,multi-function display as a passive display terminal without anyrequirement for display or graphics generation/output hardware on thePED, or any native forms of input or interactivity on the PED andwherein the function of video display provides a service to directlyconnect an abstract application to the multi-function displays fordisplay and interaction on the flight-deck, regardless of specifics,services, capabilities or operating system of the PED.
 16. The airplaneof claim 8, wherein applications displayed on flight-deck-installed,multi-function display comprise: a window on the flight-deck-installed,multi-function display, a window managed across multipleflight-deck-installed, multi-function displays, multiple application onmultiple devices, a diversity of pilot control interfaces including anability to operate without the pilot control interfaces, an abstractnature of hosted applications including a non-video application, acompositing of certified and operationally approved information, amultiple user and shared/exclusive application environment application,device-based status information sent to the gateway including additionalapplication categories, or a combination thereof.
 17. The airplane ofclaim 8, wherein the gateway comprises an application gateway and thesecurity gateway, and the aircraft interface device comprises theapplication gateway and connects the flight deck crew network to theaircraft data network, wherein the application gateway is configured toreceive the information from the PED and the security gateway isconfigured to receive information from the aircraft data network. 18.The airplane of claim 8, further comprising a graphics generator moduleconfigured to merge information from the PED with certified informationfrom an onboard system for unified presentation on theflight-deck-installed, multifunction display.
 19. The airplane of claim18, further comprising a firewall to protect the graphics generatormodule.
 20. A system comprising: a flight-deck-installed, multi-functiondisplay; a flight deck crew network; an aircraft interface deviceconnected to the flight deck crew network to couple a portableelectronic device (PED) to the flight-deck-installed, multi-functiondisplay, the aircraft interface device operable to receive informationfrom a PED hosted application in the PED; an aircraft data networkconnected to the aircraft interface device; avionics connected to theaircraft data network and the flight-deck-installed, multi-functiondisplay to display the information on the flight-deck-installed,multi-function display, wherein the avionics comprise a security gatewayconfigured to receive the information from the PED from the aircraftdata network, wherein the aircraft interface device and the securitygateway provide multiple layers of network security; and a pilot controlinterface wherein the pilot control interface and theflight-deck-installed, multi-function display are usable by a pluralityof users where each user can provide input and control their own PED andflight-deck-installed, multi-function display.
 21. The system of claim20 wherein the aircraft interface device is further operable to couplethe pilot control interface to the PED through the flight deck crewnetwork and/or the aircraft data network to enable controlling aplurality of functions on the PED via the pilot control interface. 22.The system of claim 20 where each user can provide input and exclusivelycontrol an application on another user's PED on theirflight-deck-installed, multi-function display, where each user cancontrol separate PEDs independently, and where users can cooperativelycontrol a single PED.
 23. The system of claim 20 wherein the aircraftinterface device comprises an application gateway wherein theapplication gateway or the security gateway is operable to provide afunction comprising: device authentication, bandwidth monitoring, deviceand network control, content analysis, enforcement of regulatorypolicies, enforcement of and configurability to airline policies,warning/alerting, operational status monitoring, failure handling,management of connected devices, application selection, intermediateformatting, compositing of application information, applicationauthentication, filtering of devices and applications which can orcannot communicate with the system, interfacing and data translationbetween disparate system components, relaying of aircraft informationdata and pilot interaction control data, enforcement of and compliancewith flight deck human interface standards and guidelines, enforcementof airplane manufacturer policies and acceptability as per anestablished application governance process, integration of applicationshosted on operationally approved PEDs with certified or operationallyapproved applications hosted within the flight deck crew network,preparation of information from the PED for integration or layeringwithin functional layered applications on a certified displayapplication, or a combination thereof.
 24. The system of claim 20,wherein for a function of video display, the PED utilizes theflight-deck-installed, multi-function display as a passive displayterminal without any requirement for display or graphicsgeneration/output hardware on the PED, or any native forms of input orinteractivity on the PED and wherein the function of video displayprovides a service to directly connect an abstract application to themulti-function displays for display and interaction on the flight-deck,regardless of specifics, services, capabilities or operating system ofthe PED.
 25. A non-transitory computer readable storage medium foraccessing a portable electronic device (PED) comprising instructionsstored thereon, that when executed on a processor, perform a methodfurther comprising: establishing an onboard network coupled to aflight-deck-installed, multi-function display, wherein the onboardnetwork comprises a flight deck crew network and an aircraft datanetwork; coupling the PED to the onboard network; receiving informationfrom a PED hosted application in the PED by an aircraft interfacedevice; routing the information through the aircraft interface device tothe aircraft data network; receiving the information from the aircraftdata network by a security gateway, wherein the aircraft interfacedevice and the security gateway provide multiple layers of networksecurity; displaying the information on the flight-deck-installed,multi-function display; and coupling a pilot control interface to thePED through the onboard network, wherein the coupling of the pilotcontrol interface to the PED comprises coupling a plurality of input andcontrol devices to the PED, the plurality of input and control devicesand the flight-deck-installed, multi-function display are usable by aplurality of users.
 26. The computer readable storage medium of claim 25wherein the method further comprises controlling a plurality offunctions on the PED via the pilot control interface.
 27. The computerreadable storage medium of claim 26 wherein the routing of theinformation through the aircraft interface device further comprisesproviding a function comprising: device authentication, bandwidthmonitoring, device and network control, content analysis, enforcement ofregulatory policies, enforcement of and configurability to airlinepolicies, warning/alerting, operational status monitoring, failurehandling, management of connected devices, application selection,intermediate formatting, compositing of application information,application authentication, filtering of devices and applications whichcan or cannot communicate with the onboard network, interfacing and datatranslation between disparate system components, relaying of aircraftinformation data and pilot interaction control data, enforcement of andcompliance with flight deck human interface standards and guidelines,enforcement of airplane manufacturer policies and acceptability as peran established application governance process, integration ofapplications hosted on operationally approved PEDs with certified oroperationally approved applications hosted within the onboard network,preparation of information from the PED for integration or layeringwithin functional layered applications on a certified displayapplication, or a combination thereof.
 28. An airplane comprising: afuselage portion; a wing portion; and a flight-deck-installed,multi-function display; an onboard network; an aircraft interface deviceconnected to the onboard network to couple a portable electronic device(PED) to the flight-deck-installed, multi-function display, the aircraftinterface device comprises an application gateway operable to receiveinformation from a PED hosted application in the PED and route theinformation through the application gateway; a pilot control interfacewherein the aircraft interface device is further operable to couple thepilot control interface to the PED through the onboard network to enablecontrolling a plurality of functions on the PED via the pilot controlinterface; and avionics to display the information on theflight-deck-installed, multi-function display, wherein the avionicscomprise a security gateway configured to receive the information fromthe PED, wherein the aircraft interface device and the security gatewayprovide multiple layers of network security.